Distributed Hashtable on Pre-structured Overlay Networks

Internet overlay services must adapt to the substrate network topology and link properties to achieve high performance. A common overlay structure management layer is desirable for enhancing the architectural modularity of service design and deployment. For instance, new link probing techniques can be incorporated into the common structure layer such that a large number of overlay services can benefit transparently. Additionally, a shared substrate-aware overlay structure can potentially reduce redundant per-service link-selection probing when overlay nodes participate in multiple services. The concept of building services on a common structure management layer fits well with unstructured services, those that do not place specific requirements on the overlay connectivity structure (e.g., Gnutella). Despite the benefits, it is unclear how the distributed hashtable (DHT) service can take advantage of a serviceindependent structure management layer, considering recently proposed scalable DHT protocols all employ protocol-specific overlay structures. In this paper, we present the design of a self-organizing DHT protocol based on the Landmark Hierarchy. Coupled with a simple low-latency overlay structure management protocol, this approach can support low-latency DHT lookup without any service-specific requirement on the overlay structure. Compared with Chord, a well-known DHT protocol, simulations and experimentation on 51 PlanetLab sites find that the proposed scheme can deliver better lookup performance (reducing the lookup latency by almost half) under the same link density. This benefit is achieved at the cost of less balanced lookup routing overhead. Our evaluation also demonstrates that the balance of key placement and fault tolerance for the proposed scheme are close to those of Chord. However, our approach produces more key reassignments after overlay membership changes, due to its structure-sensitive DHT mapping scheme.